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Software  Assurance  Engineering 


Mission  success  for  software-reliant  systems  requires  software 
assurance 

Software  assurance:  implementing  software  with  a  “level  of  confidence  that 
software  functions  as  intended  and  is  free  of  vulnerabilities,  either 
intentionally  or  unintentionally  designed  or  inserted  as  part  of  the  software, 
throughout  the  life  cycle...”  Section  933  of  National  Defense 
Appropriation  Act  2013 


Engineering  software  assurance  into  the  acquisition  and  development 
life  cycle 

•  Task  1:  Analyzing  Security  Risk  Early  in  the  Software  Life  Cycle 

•  Task  2:  Applying  Software  Quality  Models  to  Software  Assurance 
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Task  1  Analyzing  Security  Risk  Early  in  the  Software 
Life  Cycle 


Three  main  causes  of  operational  security  vulnerabilities: 

•  Design  weaknesses 

•  Implementation/coding  vulnerabilities 

•  System  configuration  errors 

Design  weaknesses  are  not  easily  addressed  during  operations. 

•  379  of  the  940  common  weakness  enumerations  (CWEs)  are  design 
weaknesses  (http://cwe.mitre.org/) 

•  19  of  the  top  25  are  linked  to  design  weaknesses(http://cwe. mitre.org/top25/) 
Causes  for  design  weaknesses: 

•  Poor  security  requirements 

•  Limited  understanding  of  the  impact  of  security  risk  on  mission  success 


Goal:  Reduce  Security  Design  Weaknesses  With  Improved  Requirements 
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Task  1 :  Limitations  in  Current  Security  Design  Practices 


Identification  techniques  are  ad  hoc 

•  Notation  for  expressing  a  security  event/risk  is 
incomplete 

•  Approaches  rely  on  analysts'  tacit  knowledge  of 
operational  context 

•  Security  controls  address  compliance  not  risk 

Risk  analysis  (if  done)  is  focused  on  a  single  system 

•  Standalone  (i.e.,  single  system)  models  have  been 
developed 

•  Risk  analysis  considers  the  exploit  of  an  individual 
vulnerability  within  a  single  system 


•  Security  experts  need  to  communicate  risk  effectively  with  system  and  software 
engineers  and  acquisition  experts 

•  Attacks  frequently  come  from  other  trusted  systems 

•  Complex  attacks  need  to  be  included  in  security  risk  evaluation 
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Task  1:  Security  Engineering  Risk  Analysis 
(SERA)  Framework 


Model  the  system,  context,  and  attacks 
to  determine  the  security  risks: 

•  mission  impact  of  security  breaches 

•  sufficiency  of  planned  security 

•  gaps  for  mitigation 


Workflow  /  Mission  Thread 


Mission  Degradation  / 
Produces  Mission  Failure 


Affects 


Mission  Data 
l 

i Targets 


Outcomes 

•  Disclosure  of  data  (Confidentiality) 
Modification  of  data  (Integrity) 

Produces  •  Insertion  of  false  data  (Integrity) 

•  Destruction  of  data  (Availability) 

•  Interruption  of  access  to  data  (Availability) 
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Task  1:  SERA  Approach 


Use-Case  View 


Network  View 


Physical  View 
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Threat  Identification  Models 


Data  View 


Consequence  Analysis  Models 


Workflow  View 
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Workflow  Consequences  -  Stakeholder  Consequences 

““ 1  1 

► 

Attack 

Action  1- Enablers 
Action  2 -Enablers 
Action  3 -Enablers 
Action  4 -Enablers 


Action  N- Enablers 


Stakeholder  View 


Defined  semantics  for  expressing  a  security  event/risk 
Library  of  threat  archetypes  to  support  threat  identification 
Models  to  support  threat  identification 
Models  to  support  consequence  analysis 
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Task  1:  SERA  4-Step  Process 


Risk  Identification  Worksheet 
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Task  2  Applying  Software  Quality  Models  to 
Software  Assurance 


The  SEI  has  quality  data  for  over  100 
Team  Software  Process  (TSP) 
development  projects  used  to  predict 
operational  quality. 

Data  from  five  projects  with  low 
defect  density  in  system  testing 
reported  very  low  or  zero  safety 
critical  and  security  defects  in 
production  use. 


Goal:  Use  Predictions  of  Quality  to  Inform  Security  Risk  Predictions 
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Quality  Focus  on  Defect  Injection  and  Removal 
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Removal^Defec  ts  ‘Yield 


Vulnerabilities  are  defects 

•  1-5%  of  quality  defects  are  vulnerabilities  (confirmed) 

•  50-70%  of  security  vulnerabilities  are  quality  defects  (unconfirmed) 
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Task  2:  Evidence  Available  for  Quality  Analysis 


The  data  below  describe  successful  results  of  systems  in  operation  for  at  least  a 
year  from  three  organizations  that  are  of  interest  to  identify  patterns  appropriate  for 
security. 


Org. 

Project 

Type 

Secure  or  Safety 
Critical  Defects 

Defect 

Density 

Size 
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1.3  MLOC 
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T1 
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20.00 

.1  MLOC 

With  one  exception,  projects  implemented  below  20 
defects  per  MLOC  had  no  reported  operational 
security  or  safety-critical  defects. 

The  exception  utilized  specialized  defect  removal 
practices  for  secure  systems. 
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Post  Release  Safety-Critical 
or  Security  Defects 
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Task  2:  How  Could  Quality  Help  Security? 


Good  quality  will  ensure  proper  implementation  of  specified  results 

•  Effective  code  checking  will  identify  improper  implementations  of 
specifications  (1 1  of  SANS  Top  25) 


•  Effective  design  reviews  will  identify  missing  requirements  (12  of  SANS 
Top  25) 

-  if  appropriate  security  results  are  considered  in  the  development  of 
requirements 

-  if  requirements  are  effectively  translated  into  detail  designs  and  code 
specifications  to  support  the  required  security  results 

SANS  Top  25:  SysAdmin,  Audit,  Network,  Security  Top  25  Most  Dangerous  Programming  Errors 
(http://cwe.mitre.org/top25) 


Security  Requirements  Must  be  Properly  Specified 
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Successful  Projects  Embed  Quality  and 
Safety/Security  Inspection  at  Each  Lifecycle  Step 


Design 


Successful  Projects  Use  Metrics  Extensively 


Development  Metrics 

•  Incoming/week 

•  T riage  rate 

•  %  closed 

•  Development  work  for  cycle 

•  Software  change  request  per  developer  per  week 

•  #  developers 

•  Software  change  request  per  verifier  &  validator  per  week 

•  #  verification  persons 
Software  Change  Metrics 

•  Fixed  work  per  cycle 

•  Deferred  planned  work  per  cycle 


Testing  and  tools  are  used  to  confirm  results  NOT  find  problems 
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Successful  Projects  Show  Improved  Reliability 


Software  Release 
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FY  14-03  Deliverables 


Publications: 

•  CrossTalk  September/October  2014  “Evaluating  Security  Risks  Using  Mission 
Threads”  provides  a  summary  of  the  SERA  framework  with  an  example 

•  SEI  Special  Report  CMU/SEI-2013-SR-018  “Wireless  Emergency  Alerts  (WEA) 
Cybersecurity  Risk  Management  Strategy  for  Alert  Originators”  released  March  2014 
http://resources.sei.cmu.edu/librarv/asset-view.cfm?assetid=70071 

•  Two  technical  notes  (drafts  are  in  review)  to  be  released  in  November 

Conferences: 

•  Presentation:  “Modeling  Early  Life  Cycle  Security  Risk”  International  Conference  on 
Conceptual  Modeling  2014  (ER2014)  http://2014.erconference.org/ 

•  Tutorial:  “Security  Risk  Management  using  the  Security  Engineering  Risk  Analysis 
(SERA)  Method”  2014  Annual  Computer  Security  Applications  Conference  (ACSAC) 
https://www.acsac.org/2014/ 

•  Proposed  tutorial  for  Ground  Systems  Architecture  Workshop  (GSAW)  2015 

•  Abstract  approved  for  IEEE  International  Symposium  for  Homeland  Security  2015 
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FY14-03  Collaborations 


Carnegie  Mellon  University  funded  collaboration  with  Travis  Breaux, 
Assistant  Professor  of  Computer  Science,  and  his  doctoral  student 
Hunan  Hibshi  for  academic  review  and  two  publications: 

•  H.  Hibshi,  T.D.  Breaux,  M.  Riaz,  L.  Williams  (2014)  "Towards  a  Framework  to  Measure 
Security  Expertise  in  Requirements  Analysis,”  In  Proc.  IEEE  1st  International  Workshop 
on  Evolving  Security  and  Privacy  Requirements  Engineering  (ESPRE),  pp.  13-18. 

•  H.  Hibshi,  T.D.  Breaux,  M.  Riaz,  L.  Williams  (2014)  "Discovering  Decision-Making 
Patterns  for  Security  Novices  and  Experts,”  In  Submission:  International  Journal  of 
Secure  Software  Engineering. 

SERA  Framework  review  by  Ranjit  Singh  Mann,  PE  Software 
Engineering  IPT  Lead,  Indirect  Fire  Protection  Capability  Increment  2- 
Intercept  (IFPC  Inc  2-1)  Product  Office  (PdO) 

Research  Review  Workshop,  August  6,  2014  in  Ballston  with  16 
participants  from  DoD,  NSA,  academia,  and  industry  including  the  DoD 
Software  Assurance  Community  of  Practice  (SwA  COP)  Metrics  Team 
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Contact  Information 


Carol  Woody,  Ph.D. 

Technical  Manager 
CERT/CSF/CSE 
Telephone:  +1  412-268-5800 
Email:  info@sei.cmu.edu 

Web 

www.sei.cmu.edu 

www.sei.cmu.edu/contact.cfm 


U.S.  Mail 

Software  Engineering  Institute 
Customer  Relations 
4500  Fifth  Avenue 
Pittsburgh,  PA  15213-2612 
USA 

Customer  Relations 

Email:  info@sei.cmu.edu 
Telephone:  +1  412-268-5800 

SEI  Phone:  +1  412-268-5800 

SEI  Fax:  +1  412-268-6257 
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